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ABSTRACT 



This paper offers a new method of evaluating the 
Supplemental Instruction (SI) model as implemented in biology courses at 
National-Louis University in Chicago, Illinois. From 1990 through 1993, 14 
classes of general biology were taught by the same instructor, using the same 
textbook and syllabus. An SI leader was available for five of the classes, 
and conducted SI sessions before or after class. A total of 140 students 
attended classes without SI, while 94 students with an SI leader. It was 
found that classes with SI had an average grade of 74.1 percent, while those 
without SI had an average grade of 67.6 percent. Within the classes that had 
an SI leader, grades of students who attended SI sessions were, on average, 

12 percent higher than the grades of students who did not attend the 
sessions. (Contains 11 references.) (MDM) 
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Abstract 



Supplemental Instruction (SI) is a model of academic support that has been used 
effectively at the college level to promote higher-level thinking skills in an interactive setting. 
We analyze this model as used for a high-risk biology course on a multicultural, multiethnic 
university campus. Examination grades indicated that the average grade of participants in 
classes that had SI sessions was significantly higher than that of participants in classes where 
SI sessions were not offered. Assessing the effect of SI in this way avoids the bias of 
comparing students of the same class who attend SI sessions with students who do not attend. 
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Introduction 

In this paper we describe a new method of evaluating the Supplemental Instruction 
(SI) model as implemented in biology courses in an urban multicultural university campus. 
Classes with and without SI leaders were compared, instead of final grades of students who 
did or did not participate in SI sessions. 

Supplemental Instruction is an integrative model of student academic support that was 
developed and tested during the mid 1970's at the University of Missouri (Blanc, DeBuhr, and 
Martin, 1983), and implemented at 150 institutions of higher education (National Center for 
Supplemental Instruction [NCSI], 1994). It is a process-product paradigm that determines its 
success on the final grades of students who attend or do not attend the SI sessions. 

Garland (1987) conducted a study of students at 13 institutions to test whether SI was 
effective for minority students at both ends of the academic spectrum. The study reports that 
students in the SI group benefitted across all grade levels at a significance of 0.0001. For 
example, SI students in the lowest quartile and the highest quartile scored about 0.5 letter 
grade higher than non-Sl students in the same quartile. In addition, the rate of D, F, and 
Withdrawals among SI students dropped 7%. Blanc, DeBuhr, and Martin (1983) report 
similar results regarding the effectiveness of SI for students of different levels of academic 
preparedness by analyzing the performance of students in seven courses who scored in the top 
and bottom quartiles of college entrance tests. Approximately 30% of students in both the top 
and the bottom quartiles participated in the sessions. Students participating in SI scored about 
0.8 grades higher in both the top and the bottom quartiles than ones not participating. It also 
appeared that SI was associated with reduced D, F and W (withdrawal) grades, increased re- 
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enrollment and higher GPA. Kallison and Kenney (1992) compared the grades earned by 
students in a calculus course. Students who attended SI sessions earned between 0.1 and 0.4 
letter grades higher than students who attended conventional tutorials. The same study reports 
that lower-ability students benefitted from SI more than higher-ability students. 

The use of the SI model of academic support has been recently reported to attract 
minority students to medicine (Bridgham and Scarborough 1992), to benefit problem-solving 
in chemistry courses (Congos 1993), and to increase organizational skills and create learning 
strategies in biology (Matthews 1993). 

The present study on the effect of SI in biology was conducted over the course of 
three years at National-Louis University (NLU), a private university with several campuses in 
the Chicago area, as well as throughout the US and overseas. The population at the urban 
campus, the site of this study, has a high percentage of minority students: approximately 20% 
are African-American, 10% Hispanic, and 6% Asian. This is about three times higher than 
minority percentages in Garland's (1987) study. 

The coordinator of the SI program determined that General Biology and Introduction 
to Music were "high-risk" courses and eligible for the SI program, since fifty percent of the 
students were receiving grades of D or F, or W. Most students take General Biology to fulfill 
the science requirement of their programs. The course emphasizes the concepts of biology 
and serves as a prerequisite for all subsequent biological science courses. 

At NLU the SI leaders are students who have successfully completed the course and 
are recommended by the instructor to the SI coordinator. Moreover, most of the SI leaders in 
Biology are students intending to major in elementary education. In their role as SI leaders, 
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they attend the lectures and laboratories and serve as role models for the currently enrolled 
students. The SI coordinator trains the leaders, making sure that the SI leaders' interactions 
with the students promote higher-level thinking skills rather than a repetition of the lecture. 

The conventional method of assessing the success of the SI program is to compare the 
achievement of students who attended SI sessions with those who did not (Blanc, DeBuhr, 
and Martin, 1983; Lundeberg, 1990; Martin and Arendale, 1991; NCSI, 1994). However, 
despite the claims of the authors of the SI model (Blanc, DeBuhr, and Martin 1983), this type 
of comparison is liable to be grossly biased because the students are not randomly assigned to 
the SI or non-SI groups. Rather, they are self-selected, so it is very possible that the better 
students, who would have received the better grades in the first place, tend to be the ones 
who attend the sessions. Indeed, Visor, Johnson, and Cole (1992) reported that students who 
attended SI sessions regularly were the ones who possess high self-esteem, and found active 
participation in such sessions to be academically challenging. 

In order to avoid this bias, we assess the success of the SI sessions by comparing the 
test scores of whole classes, some, but not all, of which had an SI leader. If SI participants 
are compared with non-participants within the same class, one cannot distinguish whether 
better performance is due to SI or rather to the difference in student characteristics. Our 
approach avoids this particular bias. We assume that the groups of students of different 
classes of the same course are equivalent and may therefore be compared. The SI leaders 
were available for only some of the Biology classes, choosing a class to work with based only 
on their own schedules. 

Our approach of comparing whole classes does not permit direct inference as to which 
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students profit from SI. Not surprisingly, the influence of the SI sessions is affected by the 
degree of student attendance. Blanc, DeBuhr, and Martin (1983) reported a decrease in D, F 
and Withdrawal grades with the increase of participation in SI sessions. In our study, 
attendance at sessions was voluntary, so only some students attended the SI sessions, and 
those participating attended irregularly. We assume that the amount of participation in 
sessions increased the success of the class as a whole. It is, therefore, possible that with full 
attendance the effect of offering SI could have been even stronger than it appears from our 
data. We estimated what the full contribution of the SI would have been had all students 
participated fully. 

We tested two hypotheses: 1) That the percent of very low grades was significantly 
decreased in classes with SI, and 2) that the average grade of the whole class increased in 
the classes with SI. 

Materials and Methods 

From the Winter of 1990 through the Winter of 1993, fourteen classes of General 
Biology were taught by the same instructor, using the same textbook and the same syllabus. 
An SI leader was available for five of those classes. Classes at NLU are typically small, 
approximately 20 students; a total of 94 students attended the classes where a leader was 
available, and 140 attended classes without one. The SI leader conducted sessions 
immediately before or after the class. The instructor sometimes assisted the leader with 
lecture notes, sample tests, and any other material that might help students master the content 
of the course. The names of students that attended and the number of sessions they attended 
were monitored by the leader, but the instructor was not informed, so as to prevent bias in 
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awarding final grades. 

Students in each class took two midterms and a final exam. There were make-up tests 
available for the two midterms for students receiving less than 70%, with the higher grade 
establishing the final test grade. The final test grade was the unweighted average of the three 
test grades calculated in percentage points. The actual test grades are objective and 
comparable measures of knowledge because they are percentages of correct replies to a 
sample of questions out of the same pool. Final letter grades were not used in the present 
study because they also included points for the less objective laboratory reports, homework 
problems, and an oral report. 

For purposes of this study, we assumed that any differences in the distribution of 
grades between classes with and without SI sessions were due solely to the availability of SI. 
We computed the respective class averages, and tested the significance of the difference by 
the Mann- Whitney U-test (Siegel, 1956). In addition, we calculated and plotted the 
distribution of grades. We compared the percentage of the low grades (below 60%) in SI 
classes to that in non-SI classes, and assessed the difference by a Chi-Square test. The same 
analysis was done for the high grades (above 80%). 

For classes with an SI leader, we measured the attendance in SI sessions by means of 
an "SI Quotient," which is defined as the number of student-hours of attendance divided by 
the total number of student-hours offered (i.e., the total attendance in SI sessions divided by 
the product of the number of students in the class multiplied by the number of sessions 
offered). 
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Discussion of Results 

The average grades of the 14 classes are shown in Table 1. 
Table 1. Percentage grades of the classes (on a scale of 0-100 points) 





Day Classes 


Evening Classes 


All Classes 


Non-SI classes 
(9 classes) 


72.8, 55.3, 68.6, 77.6, 59.1, 68.5 
Average = 67.0 


68.5, 69.4, 68.3 
Average = 68.7 


67.6 


SI classes 
(5 classes) 


71.3, 79.3, 74.3, 76.4 
Average = 75.3 


69.4 

Average = 69.4 


74.1 



There is considerable variability between the classes. Although it might be expected 
that day and evening students represent different student populations, there is no striking 
difference in the grades that the two groups earned. Therefore, results from the day and 
evening classes were pooled together to test the difference between the average grades in the 
SI and non-SI classes. The classes with an SI leader had an average grade of 74.1% and 
those without a leader had an average of 67.6%, a difference of 6.5 percentage points. (These 
are unweighted averages, since the units compared were whole classes). We tested the 
difference between the set of average grades of classes with and without an SI leader was 
tested by means of the Mann- Whitney U test and found it to be significant at a 0.05 
confidence level. 

As a comparison, we calculated the effect of SI according to the conventional method 
of measuring success in that program. Within the classes that had an SI leader, grades of 
students who attended SI sessions were on average 12.0% higher than grades of students who 
did not attend the sessions (the data leading to this difference are not shown). As stated 
above, we do not feel that this is an accurate measure of the effect of SI, since the students 
who attended the sessions were self-selected and not randomly assigned. 

erJc 9 
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Table 2. Attendance in SI sessions 





Number of Students 


Number of 
Sessions 
Offered 


Total 

Attendance* 


Avg. No. of 
Sessions 
Attendedf 


SIQ* 


Attended 


Not 

Attended 


Spring 91 


7 


6 


No Data 


No Data 


No Data 




Fall 91 


12 


11 


18 


100 


8.3 


0.24 


Winter 92 


15 


5 


15 


78 


5.2 


0.26 


Spring 92 


13 


6 


18 


75 


5.8 


0.22 


Winter 93 


12 


7 


10 


38 


3.2 


0.20 



* The sum of the attendance counts for all sessions in quarter 

t Average is the total attendance divided by the number of students who attended SI sessions 

* SIQ is the SI quotient, the proportion of student-hours of attendance out of the total student-hours 
of available SI sessions 



Table 2 shows the attendance in the SI sessions. For the four classes where SI 
quotients could be calculated, the quotients were 0.20, 0.22, 0.24, and 0.26. This reflects the 
fact that many students did not attend the sessions, and only a few attended sessions 
regularly. For instance, a quotient of 0.25 could have resulted if half of the students had 
ever attended a session, and those who did attend participated in only half of the sessions 
offered. If we pool together the four classes for which the SIQ could be computed, we find 
a total attendance of 291 student-hours out of (12+11)-18 + (15+5)-15 + (13+6)-18 + 

(12+7)- 10 = 1246 student-hours of SI sessions available, yielding an average SIQ of 
291 / 1246 = 0.23. Presumably, if all students had attended all sessions, the improvement of 
the average grade would be more extensive than that observed. Most of the improvement 
would have been attained by full attendance of those students who had ever attended 
sessions, and a small part by the participation of those who had never attended sessions. 
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Assuming that the increase in the average grade is proportional to the SIQ, we 
estimate the increase to be 6.5 / 0.23 = 28.3 percentage points (average observed 
improvement divided by the average SIQ). This would raise the average grade from 67.6 to 
an "idealized grade" of 95.9 percentage points. Unfortunately, it is not feasible to test this 
proportionality assumption by performing regression analysis of the increase in grades versus 
SIQ on our data because (1) only 4 observations of SIQ are available and (2) they are too 
close (between 0.20 and 0.26) to permit a reliable fit. Thus, we regard the proportionality 
assumption to be merely a working hypothesis. 



<FIGURE 1 SHOULD BE PLACED HERE > 



The distribution of grades in classes with and without an SI leader are shown in 
Figure 1. In classes that had an SI leader there were fewer students who earned low grades 
(below 60%), and more students who earned high grades (above 80%), compared to classes 
that had no SI leader. Chi-Square tests on our sample show that in both the high and low 
groups these differences are statistically significant at a 0.01 confidence level. 

Significance of Results 

The difference between grades of whole classes that had an SI leader and those that 
did not is 6.5 percentage points. If we compare the average grades of students who attended 
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SI sessions with those of students who did not within the same class, we find a larger 
difference of 12 percentage points. This 12% difference is about twice as much as the 
improvement of up to about one half letter grade found by Garland (1987) and NCSI (1994). 
However, percentage points and letter grades are not unequivocally comparable, as the 
former are more objective. The improvement of the grades of the students who actually 
attended the SI sessions cannot be completely attributed to the SI leader. Rather, many of 
these higher grades could be attributed to the interest, motivation, time devoted to studies, 
and other characteristics of this self-selected group of students who chose to attend the 
sessions. 

Another way of assessing the effect of SI sessions on students' grades is to take into 
account the attendance in the SI sessions. Blanc, DeBuhr, and Martin (1983) showed that 
the percentage of failures and withdrawals decreases with the increase of percentage of 
attendance in SI sessions. To take into consideration the variation in the attendance we 
looked at the "SI quotient." We assumed as a working hypothesis that the contribution of 
the SI leader to improving the grades of the students is proportional to SIQ. Under this 
assumption and with full attendance of all students in all sessions (SIQ =1), the average 
grade would have reached a surprising 95.9%. It is unrealistic to expect that any amount of 
academic support can increase grades to such an extent, since this estimate exaggerates the 
possibilities of improving grades by attending SI sessions. There may be several other 
factors involved, such as the time and effort students themselves invest to bring their grades 
up to the maximum. 

In classes in which an SI leader was available, the number of students receiving 
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grades below 60% decreased, whereas the number of students receiving grades above 80% 
increased. This indicates that students can advance from the lower to the medium range of 
grades, and from the medium into the upper range. It also shows that even relatively good 
students attend SI sessions, and sheds an interesting light on who chooses to attend and who 
profits from the sessions. Students from all levels of academic potential attend and 
practically all benefit to some extent from attending SI sessions. 

There are two limitations of our study. The first is the small number of classes, 14 in 
all. The second is the typically small class size (approximately 20 students); a greater class 
size would make the class average more reliable but would hinder the students. Therefore, 
whereas the results are statistically significant, one can question how much the results can be 
generalized. We believe that with larger lecture, classes the effect of SI would be even more 
pronounced. In large classes there is less opportunity for individual attention by the 
instructor, and therefore the SI session has the potential of greater individual benefit. 

Conclusion 

Our approach to the assessment of the success of SI sessions avoids the bias of 
comparing self-selected groups. However, neither our approach of comparing the 
achievements of classes with and without an SI leader, nor the conventional method of 
comparing achievement of students who did or did not attend SI sessions, gives a full 
account of the potential of the SI model of academic support. Our approach reflects the 
effect of the leader, but certainly underestimates the maximum impact on individual students 
that may be gained from SI sessions, whereas the conventional approach exaggerates this 
impact. But both demonstrate considerably high success rates. In view of this conclusion. 
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students should be encouraged to attend in increasing numbers and more regularly. 
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Figure 1. Grade distributions in classes with and without SI leaders 
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(5 classes) 
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69.4 
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Table 2. Attendance in SI sessions 





Number of Students 


Number of 
Sessions 
Offered 


Total 

Attendance* 


Avg. No. of 
Sessions 
Attendedf 


SIQ 1 


Attended 


Not 

Attended 


Spring 91 


7 


6 


No Data 


No Data 


No Data 




Fall 91 


12 


11 


18 


100 


8.3 


0.24 


Winter 92 


15 


5 


15 


78 


5.2 


0.26 


Spring 92 


13 


6 


18 


75 


5.8 


0.22 


Winter 93 


12 


7 


10 


38 


3.2 


0.20 



* The sum of the attendance counts for all sessions in quarter 

f Average is the total attendance divided by the number of students who attended SI sessions 

* SIQ is the SI quotient, the proportion of student-hours of attendance out of the total student-hours 
of available SI sessions 





U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
Educational Resources Information Center (ERIC) 




REPRODUCTION RELEASE 

(Specific Document) 



I. DOCUMENT IDENTIFICATION: 



Title 



of jif ($0$^ /eiSr?/ 



Au tho r( s) : j^ rJL /R 



Corporate Source: 



II. REPRODUCTION RELEASE: 



i Publication Date: 

i 72/ 



n rtv ln ° ,d Z ,0 ^ S , Se 7 na,e a , S 7^ ely as P° ssible timely significant materials of interest to the educational community, documents announced 
in the monthly abs ract journal ol the ERIC system. Resources in Education (RIE), are usually made available to users in microfiche reproduced 

ai£n toTie ^r^of nh S °' d , ttlr0U9h ,he ER,C Document Reproduction Service (EDRS) or other ERIC vendors. Credit is 
9 V ' 8n ,0 1,10 source of each document, and, if reproduction release is granted, one of the following notices is affixed to the document. 

the bottonToTth^pag^ 8 " 1 ^ *° r8prOdUC0 and dissem ' na,e * 0 identified document, please CHECK ONE of the following two options and sign at 






The sample sticker shown below will be 
affixed to all Level 1 documents 




Check here » 

For Level 1 Release: 

Permitting reproduction in i 

microfiche (4' x 6" film) or j 

other ERIC archival media ! 

(e g., electronic or optical) 
and paper copy. i 




~ ' r** u E D U C A 7 1 0* » A 

ilforvatson c *: * 




The sample sticker shown below will be 
affixed to all Level 2 documents 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS 
MATERIAL IN OTHER THAN PAPER 
COPY HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 





Check here 
For Level 2 Release: 

Permitting reproduction in 
microfiche (4" x 6’ film) or 
other ERIC archival media 
(e g., electronic or optical), 
but not in paper copy. 



Level 1 



Level 2 



Documents will be processed as indicated provided reproduction quality permits. If permission 
to reproduce is granted, but neither box is checked, documents will be processed at Level 1. 



' Sign 
here-* 
please 



0 

ERIC 



thereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate 
ftetamsnl as indicated above. Reproduction from the ERIC microfiche or electronic/optical media by persons other than 
ERIC employees and its system contractors requires permission from the copyright holder. Exception is made for non-pro//f 
reproduction by libranes and other service agencies to satisfy information needs of educators in response to discrete inquiries. * 


oignaiure. >-7 . * 

COlCtCfLs (B, /CA^U-J 


Printed Name/Position/JitJe: /j * . — y 

/hvn <?. XW, 


OrganizatioiVAcfclress: V 

Akjycnd/- /m/s Ah 

S'fo'er/dM TAodd. 

JE'i/oja cfrr? . 6 0 2. G / 


Telephone: 

SV7- 97S-//0O Y2 


3S7 




Date: 



